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METHOD AND ARRANGEMENT FOR EVALUATING IMAGES 
TAKEN WITH A FUNDUS CAMERA 



CROSS-REFERENCE TO RELATED APPLICATIONS 

[000 1 ] This application claims priority of International Application 

No. PCT//EP03/02098, filed February 28. 2003, and German Application No. 102 25 

855.4, filed June 7, 2002, the complete disclosures of which are hereby incorporated by 

reference. 

BACKGROUND OF THE INVENTION 

[0002] Digital image capture and archiving systems for image-generating methods have 
become widespread in ophthalmology in recent years. At the same time, image-oriented 
medical teaching materials and documentation (atlases) are being published in digital and 
sometimes in multimedia and interactive form to an increasing extent. However, the only 
exchange of information between image capture systems and atlases, if any, is indirect. In 
particular, the user is solely responsible for the selection of relevant entries and pictorial 
examples of the atlas, i.e., the information available in the actual state of the image capture 
system (e.g., type of directly acquired images, classification of pathological changes that can 
be detected on the latter, and the like) are not used to filter the information offered by the 
atlas. On the other hand, it is not possible to expand the atlas through one's own documented 
photographs or to use documented images of the information system to add to the patient 
history. Therefore, the usefulness of these information systems for the routine work of the 
physician and the (manageable) scope of such atlases is severely limited. 

OBJECT OF THE INVENTION 

[0003] The primary object of the present invention is a method and an arrangement 
which make it possible to link an image-oriented medical information system (hereinafter: 
atlas) with a digital image capture and archiving system for ophthalmology in such a way that 
(a) it is possible to immediately access the information of the atlas while using the 
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image capture system; 

(b) information derived from the actual state of the image capture and archiving 
system (hereinafter: contextual information) can be used to compose a selection from 
information to be provided by the atlas that is relevant for the actual work of the user; 

(c) the atlas can be expanded by documented photographs provided by the user of 
the image capture and archiving system; 

(d) documented photographs from the atlas can be transferred to the electronic 
patient records of the image archiving system in order, for example, to document conformity 
to or deviation from typical clinical phenomena. 

BRIEF DESCRIPTION OF THE DRAWINGS 

r00041 In the drawings: 

f 00051 Fig. 1 is a block diagram of an image capture system in accordance with the 
invention; and 

[0006] Fig. 2 is a flow chart of steps in the method according to the invention. 

DESCRIPTION OF AN THE PREFERRED EMBODIMENTS EXAMPLE 

[0007] The model (Figure) (shown in Fig. 1) comprises an image capture system which is 
connected to a fundus camera and combined with an image archiving system (Figure) which 
is to be linked to an atlas of fundus photographs (hereinafter: retina atlas) that is installed on 
the same computer (...). By means of a program running on this computer for controlling 
the image capture system which has simultaneous access to the information of the image 
archiving system, it is possible to start the retina atlas on this computer or on a remote 
computer to which there is a network connection, to transfer contextual information and to 
initiate searches (Figure). In addition to this infrastructure, the invention claims the suitable 
selection of contextual information and the application of a method which enables a fuzzy 
search for corresponding entries in the retina atlas. 

[0008] By way of example, Fig. 1 shows the arrangement, according to the invention, 
comprising a fundus camera 1 for photographing the fundus, for example, a Zeiss FF 450+ 
fundus camera, whose photographic unit (CCD camera) has an output 2 for sending the 
photographed fundus images in direction of an evaluating unit 3 which digitizes the recorded 
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images and stores them in a memory or storage 4 (image archive). The images are examined 
by the evaluating unit 3 for determined structures or features, for example, through gray- 
value analysis and a gray-value histogram, through color analysis and a color histogram, or 
through detection of characteristic structures (example). 

[0009] A comparison operator 5 is connected to an internal or external database 6, which 
can be, for example, a retina atlas according to (reference source) as a CD-ROM or an 
internet database, and compares images recorded by the fundus camera to images that have 
already been recorded or archived on the basis of the analyzed criterion (color, gray value, 
structure). These images can be fundus images of other eyes from pre-stored archives or 
images of one and the same patient that were recorded earlier. 

[0010] Further, the comparison operator 5 can create a new data storage for the recorded 
image and use it later for purposes of comparison. This is particularly important when: 

(a) the recorded image has features that conform only partly, or not at all, to 
stored images with respect to the image analysis and it is stored for detection of a new 
pathology image; 

(b) the recorded image is to serve as a basis for a subsequent comparison with 
newly recorded images. 

[001 1] Fig. 2 schematically shows the flow of the method according to the invention 
which comprises: 

entering patient-specific information for subsequent identification of the 
recorded image that is stored together with this information; 

taking a picture with the fundus camera; 

importing one or more pre-stored images from internal or external storage 
media (computer, CD-ROM, Internet); 

producing contextual information through image analysis of at least the 
recorded image; 

analyzing pre-stored image according to the same or similar criteria to form 
contextual information of the pre-stored images. This can also be carried out by averaging a 
plurality of images or image groups for generating standard contextual information; 

comparing the recorded images with pre-stored images by comparing the 
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contextual information for determining a diagnosis or classifying the recorded image; 
(retrieving similar images); and 

storing (new entry) the contextual information that is associated with the 
recorded image through comparison together with the image for correlating with a diagnosis 
or classification! [;]]. 

[0012] According to the invention, an image analysis for classification and for forming 
contextual information is also carried out for data that are purchased or viewable on the 
Internet (Zeiss Retina Atlas: http://www.zeiss.de/czj/de/op/zeiss/index_frames.html). 

Compilation of contextual information: 

[0013] Contextual information is drawn from 

the evaluation of the settings of the fundus camera (recording mode, field 
angle, exposure settings) that are either actually taken from the connected camera or are taken 
as entries from the image archive; 

manual annotations associated with the picture (keywords, diagnostic codes, 
pictorial elements inserted in determined positions in the picture); 

the evaluation of patient-specific information (age, sex, anamnesis, etc.); 

the evaluation of the image content. 

[0014] The last item comprises a large number of general methods that can be used for 
any image contents and specific methods for detecting and analyzing typical objects and 
changes in the ocular fundus. The first method class (see also [1], [2], [3]) includes: 

determination of color histograms and parameters derived therefrom; 

evaluation of spatial distribution of determined color values or gray values. 

[0015] The second method class (see [4], [5], [6]) comprises, for example: 

extraction of the vascular network and derivation of characteristic quantities 

(e.g., length ratio of large to small vessels, degree of arborization); 

classification and quantification of structures at the ocular fundus (e.g., 

papilla, fovea); and 

detection and quantification of pathological changes (e.g., position and 

extension of exudates, microaneurysms, scars or neovascularizations) which can be carried 
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out depending on the determined fundus camera settings. 

[0016] Accordingly, a set of attributes that encompasses the actual field of interest for 
inquiries of the retina atlas can be produced either for the actually selected image (the results 
of the image capture or of a query of the image archive) or for the actually selected patient 
(by evaluating and combining this information for a plurality of recordings), e.g., in the 
following form: 



Age 


58 


Sex 


male 


Anamnesis 


diabetes II 


Image type 


color 


Pathology 


15 microaneurysms in 3 quadrants, 

average diameter xmm, v 

9 hard exudates, total surface area xmm 2 



Table 1 : Possible attribute vector for a fundus recording 



Search method in retina atlas: 

[0017] For the attributes given above, images that belong to the same topic range are 
searched from the retina atlas and ordered according to similarity so that the user does not 
have to search manually through a large number of images. To take the example of the 
attribute vector according to Table 1, all pictures are found for nonproliferative diabetic 
retinopathy. This presupposes that corresponding attributes have already been determined for 
all of the images acquired in the retina atlas. The degrees of similarity to the given attribute 
vector can then be determined and a correspondingly sorted amount of image hits or a chapter 
or section of the retina atlas receiving the most hits can be returned by means of hierarchical 
search methods ([7], [8]). 

[001 8] The corresponding images (or the corresponding chapter) are loaded and displayed 
in the retina atlas. 
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Expansion of the retina atlas by self-prepared images: 

[0019] The user of the image capture and archiving system has the possibility of 
annotating self-prepared images (or images taken from the image archive) and adding them to 
the retina atlas by incorporating them in existing chapters or by creating new chapters or 
sections. When the images are transferred, the attributes are generated automatically and the 
image index is updated so that these new images are available for future searches in the retina 
atlas. 

Importing images and comments from the retina atlas: 

[0020] It is possible for the user to transfer selected images from the retina atlas to patient 
files in the image archiving system for purposes of documentation by means of an operator 
control function of the retina atlas. 

[002 1 ] References of the Relevant Art 

[1] Yamamoto et al. 5 "Extraction of Object Features and Its Application to Image 
Retrieval", Trans, of IEICE, vol. E72, No. 6, 771-781 (June 1989). 

[2] M. Kurokawa, "An Approach to Retrieving Images by Using their Pictorial Features", 
IBM Research, Japan, September 1989. 

[3] Gudivada, V. N., Raghavan, V. V. (editors), "Content-based image retrieval systems", 
IEEE Computer 28 (9), 18-22 (1995). 

[4] Kirkpatrick et al., "Quantitative Image Analysis of Macular Drusen from Fundus 
Photographs and Scanning Laser Ophthalmoscope Images", Eye (9) 48-55, 1995. 
[5] S. Feman et al., "A Quantitative System to Evaluate Diabetic Retinopathy from 
Fundus Photographs", Investigative Ophthalmology and Visual Science, (36): 174-180, 1995. 
[6] E. Peli, M. Lahav, "Drusen Measurement from Fundus Photographs Using Computer 
Image Analysis", Ophthalmology 93:1575-1580, 1986. 

[7] Hanan Samet, "The Quadtree and related Hierarchical Data Structures", Computing 
Surveys, vol. 16, No. 2, June 1984. 

[8] S. Berchthold et al., "The X-Tree: An Index structure for high-dimensional data", 

Proceedings of the International Conference on Very Large Databases, 28-29, 1996. 

[9] E. Petrakis, C. Faloutsos, "Similarity searching in medical image databases", IEEE 



MARKED-UP/BOLDED SUBSTITUTE SPECIFICATION and ABSTRACT 



Trans. Knowledge and Data Engineering, 9(3):435-447, 1997. 

[0022] Patents: 

US 5579471 
US 5852823 
US 5913205 
US 5911139 

[00231 While the foregoing description and drawings represent the present 
invention, it will be obvious to those skilled in the art that various changes may be 
made therein without departing from the true spirit and scope of the present invention. 
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ABSTRACT 

[0024] Method and arrangement for evaluating images recorded with a fundus camera, 
wherein deviations from a stored comparison image and/or from a standard image created by 
evaluating a plurality of comparison images are determined, and/or a similarity analysis is 
carried out by means of a stored comparison image and/or by means of a standard image 
created by evaluating a plurality of comparison images. 
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